The role of driver head pose dynamics and instantaneous driving in safety critical events: Application of computer vision in naturalistic driving.

This paper investigates the role of driver behavior especially head pose dynamics in safety-critical events (SCEs). Using a large dataset collected in a naturalistic driving study, this paper analyzes the head pose dynamics and driving behavior in moments leading up to crashes or near-crashes. The study uses advanced computer vision and mixed logit modeling techniques to identify patterns and relationships between drivers' head pose dynamics and crash involvement. The results suggest that driver-head pose dynamics, especially poses that indicate distraction and movement volatility, are important factors that can contribute to undesirable safety outcomes. Marginal effects show that angular deviation for head pose dynamics indicated by yaw, pitch and roll increase the likelihood of crash intensity by 4.56%, 4.92% and 8.26% respectively. Furthermore, traffic flow and lane changing also contribute to increase in likelihood of crash intensity. These findings provide new insights into pre-crash factors, especially human factors and safety-critical events. The study highlights the importance of considering human factors in designing driver assistance systems and developing safer vehicles. This research contributes by examining naturalistic driving data at the microscopic level with early detection of behaviors that lead to SCEs and provides a basis for future research on automation.

Investigating the relation between instantaneous driving decisions and safety critical events in naturalistic driving environment.

The availability of large-scale naturalistic driving data provides enormous opportunities for studying relationships between instantaneous driving decisions prior to involvement in safety critical events (SCEs). This study investigates the role of driving instability prior to involvement in SCEs. While past research has studied crash types and their contributing factors, the role of pre-crash behavior in such events has not been explored as extensively. The research demonstrates how measures and analysis of driving volatility can be leading indicators of crashes and contribute to enhancing safety. Highly detailed microscopic data from naturalistic driving are used to provide the analytic framework to rigorously analyze the behavioral dimensions and driving instability that can lead to different types of SCEs such as roadway departures, rear end collisions, and sideswipes. Modeling results reveal a positive association between volatility and involvement in SCEs. Specifically, increases in both lateral and longitudinal volatilities represented by Bollinger bands and vehicular jerk lead to higher likelihoods of involvement in SCEs. Further, driver behavior related factors such as aggressive driving and lane changing also increases the likelihood of involvement in SCEs. Driver distraction, as represented by the duration of secondary tasks, also increases the risk of SCEs. Likewise, traffic flow parameters play a critical role in safety risk. The risk of involvement in SCEs decreases under free flow traffic conditions and increases under unstable traffic flow. Further, the model shows prediction accuracy of 88.1 % and 85.7 % for training and validation data. These results have implications for proactive safety and providing in-vehicle warnings and alerts to prevent the occurrence of such SCEs.

Exploring microscopic driving volatility in naturalistic driving environment prior to involvement in safety critical events-Concept of event-based driving volatility.

The sequence of instantaneous driving decisions and its variations, known as driving volatility, prior to involvement in safety critical events can be a leading indicator of safety. This study focuses on the component of "driving volatility matrix" related to specific normal and safety-critical events, named "event-based volatility." The research issue is characterizing volatility in instantaneous driving decisions in the longitudinal and lateral directions, and how it varies across drivers involved in normal driving, crash, and/or near-crash events. To explore the issue, a rigorous quasi-experimental study design is adopted to help compare driving behaviors in normal vs unsafe outcomes. Using a unique real-world naturalistic driving database from the 2nd Strategic Highway Research Program (SHRP), a test set of 9593 driving events featuring 2.2 million temporal samples of real-world driving are analyzed. This study features a plethora of kinematic sensors, video, and radar spatiotemporal data about vehicle movement and therefore offers the opportunity to initiate such exploration. By using information related to longitudinal and lateral accelerations and vehicular jerk, 24 different aggregate and segmented measures of driving volatility are proposed that captures variations in extreme instantaneous driving decisions. In doing so, careful attention is given to the issue of intentional vs. unintentional volatility. The volatility indices, as leading indicators of near-crash and crash events, are then linked with safety critical events, crash propensity, and other event specific explanatory variables. Owing to the presence of unobserved heterogeneity and omitted variable bias, fixed- and random-parameter discrete choice models are developed that relate crash propensity to unintentional driving volatility and other factors. Statistically significant evidence is found that driver volatilities in near-crash and crash events are significantly greater than volatility in normal driving events. After controlling for traffic, roadway, and unobserved factors, the results suggest that greater intentional volatility increases the likelihood of both crash and near-crash events. A one-unit increase in intentional volatility is associated with positive vehicular jerk in longitudinal direction increases the chance of crash and near-crash outcome by 15.79 and 12.52 percentage points, respectively. Importantly, intentional volatility in positive vehicular jerk in lateral direction has more negative consequences than intentional volatility in positive vehicular jerk in longitudinal direction. Compared to acceleration/deceleration, vehicular jerk can better characterize the volatility in microscopic instantaneous driving decisions prior to involvement in safety critical events. Finally, the magnitudes of correlations exhibit significant heterogeneity, and that accounting for the heterogeneous effects in the modeling framework can provide more reliable and accurate results. The study demonstrates the value of quasi-experimental study design and big data analytics for understanding extreme driving behaviors in safe vs. unsafe driving outcomes.

Automated detection of microaneurysms using scale-adapted blob analysis and semi-supervised learning.

Despite several attempts, automated detection of microaneurysm (MA) from digital fundus images still remains to be an open issue. This is due to the subtle nature of MAs against the surrounding tissues. In this paper, the microaneurysm detection problem is modeled as finding interest regions or blobs from an image and an automatic local-scale selection technique is presented. Several scale-adapted region descriptors are introduced to characterize these blob regions. A semi-supervised based learning approach, which requires few manually annotated learning examples, is also proposed to train a classifier which can detect true MAs. The developed system is built using only few manually labeled and a large number of unlabeled retinal color fundus images. The performance of the overall system is evaluated on Retinopathy Online Challenge (ROC) competition database. A competition performance measure (CPM) of 0.364 shows the competitiveness of the proposed system against state-of-the art techniques as well as the applicability of the proposed features to analyze fundus images.

Retina image analysis and ocular telehealth: the Oak Ridge National Laboratory-Hamilton Eye Institute case study.

Automated retina image analysis has reached a high level of maturity in recent years, and thus the question of how validation is performed in these systems is beginning to grow in importance. One application of retina image analysis is in telemedicine, where an automated system could enable the automated detection of diabetic retinopathy and other eye diseases as a low-cost method for broad-based screening. In this work, we discuss our experiences in developing a telemedical network for retina image analysis, including our progression from a manual diagnosis network to a more fully automated one. We pay special attention to how validations of our algorithm steps are performed, both using data from the telemedicine network and other public databases.

Statistical atlas based exudate segmentation.

Diabetic macular edema (DME) is characterized by hard exudates. In this article, we propose a novel statistical atlas based method for segmentation of such exudates. Any test fundus image is first warped on the atlas co-ordinate and then a distance map is obtained with the mean atlas image. This leaves behind the candidate lesions. Post-processing schemes are introduced for final segmentation of the exudate. Experiments with the publicly available HEI-MED data-set shows good performance of the method. A lesion localization fraction of 82.5% at 35% of non-lesion localization fraction on the FROC curve is obtained. The method is also compared to few most recent reference methods.

Validating retinal fundus image analysis algorithms: issues and a proposal.

This paper concerns the validation of automatic retinal image analysis (ARIA) algorithms. For reasons of space and consistency, we concentrate on the validation of algorithms processing color fundus camera images, currently the largest section of the ARIA literature. We sketch the context (imaging instruments and target tasks) of ARIA validation, summarizing the main image analysis and validation techniques. We then present a list of recommendations focusing on the creation of large repositories of test data created by international consortia, easily accessible via moderated Web sites, including multicenter annotations by multiple experts, specific to clinical tasks, and capable of running submitted software automatically on the data stored, with clear and widely agreed-on performance criteria, to provide a fair comparison.

Exudate-based diabetic macular edema detection in fundus images using publicly available datasets.

Diabetic macular edema (DME) is a common vision threatening complication of diabetic retinopathy. In a large scale screening environment DME can be assessed by detecting exudates (a type of bright lesions) in fundus images. In this work, we introduce a new methodology for diagnosis of DME using a novel set of features based on colour, wavelet decomposition and automatic lesion segmentation. These features are employed to train a classifier able to automatically diagnose DME through the presence of exudation. We present a new publicly available dataset with ground-truth data containing 169 patients from various ethnic groups and levels of DME. This and other two publicly available datasets are employed to evaluate our algorithm. We are able to achieve diagnosis performance comparable to retina experts on the MESSIDOR (an independently labelled dataset with 1200 images) with cross-dataset testing (e.g., the classifier was trained on an independent dataset and tested on MESSIDOR). Our algorithm obtained an AUC between 0.88 and 0.94 depending on the dataset/features used. Additionally, it does not need ground truth at lesion level to reject false positives and is computationally efficient, as it generates a diagnosis on an average of 4.4s (9.3s, considering the optic nerve localisation) per image on an 2.6 GHz platform with an unoptimised Matlab implementation.

Detecting flash artifacts in fundus imagery.

In a telemedicine environment for retinopathy screening, a quality check is needed on initial input images to ensure sufficient clarity for proper diagnosis. This is true whether the system uses human screeners or automated software for diagnosis. We present a method for the detection of flash artifacts found in retina images. We have collected a set of retina fundus imagery from February 2009 to August 2011 from several clinics in the mid-South region of the USA as part of a telemedical project. These images have been screened with a quality check that sometimes omits specific flash artifacts, which can be detrimental for automated detection of retina anomalies. A multi-step method for detecting flash artifacts in the center area of the retina was created by combining characteristic colorimetric information and morphological pattern matching. The flash detection was tested on a dataset of 5218 images representative of the population. The system achieved a sensitivity of 96.54% and specificity of 70.16% for the detection of the flash artifacts. The flash artifact detection can serve as a useful tool in quality screening of retina images in a telemedicine network. The detection can be expected to improve automated detection by either providing special handling for these images in combination with a flash mitigation or removal method.

A health insurance portability and accountability act-compliant ocular telehealth network for the remote diagnosis and management of diabetic retinopathy.

In this article, we present the design and implementation of a regional ocular telehealth network for remote assessment and management of diabetic retinopathy (DR), including the design requirements, network topology, protocol design, system work flow, graphics user interfaces, and performance evaluation. The Telemedical Retinal Image Analysis and Diagnosis Network is a computer-aided, image analysis telehealth paradigm for the diagnosis of DR and other retinal diseases using fundus images acquired from primary care end users delivering care to underserved patient populations in the mid-South and southeastern United States.

Textureless macula swelling detection with multiple retinal fundus images.

Retinal fundus images acquired with nonmydriatic digital fundus cameras are versatile tools for the diagnosis of various retinal diseases. Because of the ease of use of newer camera models and their relatively low cost, these cameras can be employed by operators with limited training for telemedicine or point-of-care (PoC) applications. We propose a novel technique that uses uncalibrated multiple-view fundus images to analyze the swelling of the macula. This innovation enables the detection and quantitative measurement of swollen areas by remote ophthalmologists. This capability is not available with a single image and prone to error with stereo fundus cameras. We also present automatic algorithms to measure features from the reconstructed image, which are useful in PoC automated diagnosis of early macular edema, e.g., before the appearance of exudation. The technique presented is divided into three parts: first, a preprocessing technique simultaneously enhances the dark microstructures of the macula and equalizes the image; second, all available views are registered using nonmorphological sparse features; finally, a dense pyramidal optical flow is calculated for all the images and statistically combined to build a naive height map of the macula. Results are presented on three sets of synthetic images and two sets of real-world images. These preliminary tests show the ability to infer a minimum swelling of 300 µm and to correlate the reconstruction with the swollen location.

Automated diagnosis of retinopathy by content-based image retrieval.

PURPOSE: To describe a novel computer-based image analysis method that is being developed to assist and automate the diagnosis of retinal disease. METHODS: Content-based image retrieval is the process of retrieving related images from large database collections using their pictorial content. The content feature list becomes the index for storage, search, and retrieval of related images from a library based upon specific visual characteristics. Low-level analyses use feature description models and higher-level analyses use perceptual organization and spatial relationships, including clinical metadata, to extract semantic information. RESULTS: We defined, extracted, and tested a large number of region- and lesion-based features from a dataset of 395 retinal images. Using a statistical hold-one-out method, independent queries for each image were submitted to the system and a diagnostic prediction was formulated. The diagnostic sensitivity for all stratified levels of age-related macular degeneration ranged from 75% to 100%. Similarly, the sensitivity of detection and accuracy for proliferative diabetic retinopathy ranged from 75% to 91.7% and for nonproliferative diabetic retinopathy, ranged from 75% to 94.7%. The overall purity of the diagnosis (specificity) for all disease states in the dataset was 91.3%. CONCLUSIONS: The probabilistic nature of content-based image retrieval permits us to make statistically relevant predictions regarding the presence, severity, and manifestations of common retinal diseases from digital images in an automated and deterministic manner.

Retina lesion and microaneurysm segmentation using morphological reconstruction methods with ground-truth data.

In this work we report on a method for lesion segmentation based on the morphological reconstruction methods of Sbeh et. al. We adapt the method to include segmentation of dark lesions with a given vasculature segmentation. The segmentation is performed at a variety of scales determined using ground-truth data. Since the method tends to over-segment imagery, ground-truth data was used to create post-processing filters to separate nuisance blobs from true lesions. A sensitivity and specificity of 90% of classification of blobs into nuisance and actual lesion was achieved on two data sets of 86 images and 1296 images.

Using a patient image archive to diagnose retinopathy.

Diabetes has become an epidemic that is expected to impact 365 million people worldwide by 2025. Consequently, diabetic retinopathy is the leading cause of blindness in the industrialized world today. If detected early, treatments can preserve vision and significantly reduce debilitating blindness. Through this research we are developing and testing a method for automating the diagnosis of retinopathy in a screening environment using a patient archive and digital fundus imagery. We present an overview of our content-based image retrieval (CBIR) approach and provide performance results for a dataset of 98 images from a study in Canada when compared to an archive of 1,355 patients from a study in the Netherlands. An aggregate performance of 89% correct diagnosis is achieved, demonstrating the potential of automated, web-based diagnosis for a broad range of imagery collected under different conditions and with different cameras.

Detection of anatomic structures in human retinal imagery.

The widespread availability of electronic imaging devices throughout the medical community is leading to a growing body of research on image processing and analysis to diagnose retinal disease such as diabetic retinopathy (DR). Productive computer-based screening of large, at-risk populations at low cost requires robust, automated image analysis. In this paper we present results for the automatic detection of the optic nerve and localization of the macula using digital red-free fundus photography. Our method relies on the accurate segmentation of the vasculature of the retina followed by the determination of spatial features describing the density, average thickness, and average orientation of the vasculature in relation to the position of the optic nerve. Localization of the macula follows using knowledge of the optic nerve location to detect the horizontal raphe of the retina using a geometric model of the vasculature. We report 90.4% detection performance for the optic nerve and 92.5% localization performance for the macula for red-free fundus images representing a population of 345 images corresponding to 269 patients with 18 different pathologies associated with DR and other common retinal diseases such as age-related macular degeneration.

A probabilistic framework for content-based diagnosis of retinal disease.

Diabetic retinopathy is the leading cause of blindness in the working age population in the industrialized world. Computer assisted analysis has the potential to assist in the early detection of diabetes by regular screening of large populations. The widespread availability of digital fundus cameras today is leading to the accumulation of large image archives of diagnosed patient data that captures historical knowledge of retinal pathology. Through this research we are developing a content-based image retrieval method to verify our hypothesis that retinal pathology can be identified and quantified from visually similar retinal images in an image archive. We will present diagnostic results for specificity and sensitivity on a population of 395 fundus images representing the normal fundus and 14 stratified disease states.

Locating the optic nerve in retinal images: comparing model-based and Bayesian decision methods.

In this work we compare two methods for automatic optic nerve (ON) localization in retinal imagery. The first method uses a Bayesian decision theory discriminator based on four spatial features of the retina imagery. The second method uses a principal component-based reconstruction to model the ON. We report on an improvement to the model-based technique by incorporating linear discriminant analysis and Bayesian decision theory methods. We explore a method to combine both techniques to produce a composite technique with high accuracy and rapid throughput. Results are shown for a data set of 395 images with 2-fold validation testing.